Low Complexity Regularization of Linear Inverse Problems

Inverse problems and regularization theory is a central theme in contemporary signal processing, where the goal is to reconstruct an unknown signal from partial indirect, and possibly noisy, measurements of it. A now standard method for recovering the unknown signal is to solve a convex optimization problem that enforces some prior knowledge about its structure. This has proved efficient in many problems routinely encountered in imaging sciences, statistics and machine learning. This chapter delivers a review of recent advances in the field where the regularization prior promotes solutions conforming to some notion of simplicity/low-complexity. These priors encompass as popular examples sparsity and group sparsity (to capture the compressibility of natural signals and images), total variation and analysis sparsity (to promote piecewise regularity), and low-rank (as natural extension of sparsity to matrix-valued data). Our aim is to provide a unified treatment of all these regularizations under a single umbrella, namely the theory of partial smoothness. This framework is very general and accommodates all low-complexity regularizers just mentioned, as well as many others. Partial smoothness turns out to be the canonical way to encode low-dimensional models that can be linear spaces or more general smooth manifolds. This review is intended to serve as a one stop shop toward the understanding of the theoretical properties of the so-regularized solutions. It covers a large spectrum including: (i) recovery guarantees and stability to noise, both in terms of $\ell^2$-stability and model (manifold) identification; (ii) sensitivity analysis to perturbations of the parameters involved (in particular the observations), with applications to unbiased risk estimation ; (iii) convergence properties of the forward-backward proximal splitting scheme, that is particularly well suited to solve the corresponding large-scale regularized optimization problem

Fast Computation of Auxiliary Quantities in Local Polynomial Regression

We investigate the extension of binning methodology to fast computation of several auxiliary quantities that arise in local polynomial smoothing. Examples include degrees of freedom measures, cross-validation functions, variance estimates and exact measures of error. It is shown that the computational effort required for such approximations is of the same order of magnitude as that required for a binned local polynomial smooth. KEY WORDS: Binning; Cross-validation; Error degrees of freedom ; Kernel estimator; Linear smoother; Mean average squared error; Smoother matrix; Standard error. 1 Introduction Fast computational methods for local polynomial kernel regression have received considerable attention in the recent literature. Witness the work of Cleveland and Grosse (1991), Hardle and Scott (1992), Fan and Marron (1994), Seifert, Brockmann, Engel and Gasser (1994), Loader (1994), and Wand (1994). In each of these papers the main focus of attention has been on fast computation of the ..

Nonparametric density deconvolution by weighted kernel estimators

10.1007/s11222-008-9086-7Statistics and Computing193217-22

Evaluation of spatiotemporal imputations for fishing catch rate standardisation

As commercial fishing activity shifts to target different grounds over time, spatial gaps can be created in catch rate data and lead to biases in derived indices of fish abundance. Imputation has been shown to reduce such biases. In this study, the relative performance of several imputation methods was assessed using simulated catch rate datasets. Simulations were carried out for three fish stocks targeted by a commercial hook and line fishery off the south-western coast of Australia: Snapper (Chrysophrys auratus), West Australian Dhufish (Glaucosoma hebraicum), and Baldchin Groper (Choerodon rubescens). For High Growth scenarios, the mean squared errors (MSEs) of Geometric and Linear imputations were lower, indicating higher accuracy and precision, than Base method (constant value) imputations. For Low Growth scenarios, the lowest MSEs were achieved for Base method imputations. However, for the final standardised and imputed abundance indices, the Base method index consistently demonstrated the largest biases. Results demonstrate the importance of selecting an appropriate imputation method when standardising catch rates from a commercial fishery that changed its spatial pattern of fishing over time

Bagging in the Presence of Outliers

We investigate the performance of bagging methods in the presence of outliers. The results are best illustrated and intuitively explained for classical classification problems to which we shall restrict our focus in this paper. It is shown that bagging methods can improve the resistance of classification rules to outlier contamination, especially if an m-out-of-n bagging scheme is used. However, the outlier-reduction property does not improve performance to such an extent that outliers are no longer noticeable. It is also shown that in the absence of contamination by outliers the effects of bagging are negligible. Therefore, when bagging is not really needed, deleterious effects that result from employing it are quite small. 1 Introduction The method of bootstrap aggregation, or bagging, was introduced by Breiman (1996a). Recent accounts of its properties, in the settings of both prediction and classification, include those of Breiman (1996b) and Tibshirani (1996). When used for class..

A New Approach to Variable Selection in Least Squares Problems

The title Lasso has been suggested by Tibshirani [7] as a colourful name for a technique of variable selection which requires the minimization of a sum of squares subject to an ll bound r; on the solution. This forces zero components in the minimizing solution for small values of r;. Thus this bound can function as a selection parameter. This paper makes two contributions to computational problems associated with implementing the Lasso: (1) a com- pact descent method for solving the constrained problem for a particular value of r; is formulated, and (2) a homotopy method, in which the constraint bound r; becomes the homotopy parameter, is developed to completely describe the possible selection regimes. Both algorithms have a finite termination property

Knot Selection for Regression Splines via the LASSO

: Tibshirani (1996) proposes the "Least Absolute Shrinkage and Selection Operator" (lasso) as a method for regression estimation which combines features of shrinkage and variable selection. In this paper we present an algorithm that allows efficient calculation of the lasso estimator. In particular our algorithm can also be used when the number of variables exceeds the number of observations. This algorithm is then applied to the problem of knot selection for regression splines. 1 Introduction The performance of regression spline smoothing is governed by the choice of knots used in calculating the estimator, and much research effort has been devoted to the difficult problem of knot selection (see, e.g., Wand, 1997; Denison et al., 1998). Knot selection is not unlike variable selection in linear regression, for which Tibshirani (1996) proposes the least absolute shrinkage and selection operator. The lasso estimator is the solution of the constrained estimation problem minimise fi2R ..

Exercise-induced pulmonary haemorrhage in Thoroughbred racehorses: A longitudinal study

Background Exercise‐induced pulmonary haemorrhage (EIPH) is considered a progressive disease based on histopathology, but it is unknown if tracheobronchoscopic EIPH severity worsens over time. Objectives The aim of this study was to examine tracheobronchoscopic EIPH changes over time in a population of Thoroughbred racehorses. A secondary aim was to identify factors that affect changes in tracheobronchoscopic EIPH severity between observations. Study design Prospective, longitudinal, observational cross‐sectional study. Methods Thoroughbred racehorses were examined with tracheobronchoscopy no earlier than 30 min after racing. Examinations were recorded and graded blindly by experienced veterinarians using a 0–4 scale. Horses with 2 or more observations were included in the analysis. The association between the previous and current EIPH score was investigated using a linear mixed effect model. Factors associated with transitioning from a lower to a high EIPH grade and vice versa were examined using multiple ordinal regression. A semi‐parametric regression model was used to examine progression using the number of career starts as a marker for time. Models were adjusted for potential confounding variables. Results There were 2974 tracheobronchoscopic examinations performed on 747 horses. Blood was detected in over half of all examinations (55.6%). The population prevalence of EIPH increased as the number of examinations for each horse increased. The preceding EIPH score was significantly associated with the current EIPH score. Significant variables associated with moving between EIPH grades were the number of days since last racing, ambient temperature and weight carried. Tracheobronchoscopic EIPH is mildly progressive over the first thirty career starts. Main limitations Enrolment was voluntary. Horses were not followed for their entire career. Conclusion Limiting the number of days in the current racing preparation and spacing races for horses with moderate to severe EIPH may be beneficial for reducing tracheobronchoscopic EIPH severity. The association between ambient temperature and EIPH warrants further investigation

Toward the discrimination of early melanoma from common and dysplastic nevus using fiber optic diffuse reflectance spectroscopy

We describe a study of the discrimination of early melanoma from common and dysplastic nevus using fiber optic diffuse reflectance spectroscopy. Diffuse reflectance spectra in the wavelength range 550 to 1000 nm are obtained using 400-µm core multimode fibers arranged in a six-illumination-around-one- collection geometry with a single fiber-fiber spacing of 470 µm. Spectra are collected at specific locations on 120 pigmented lesions selected by clinicians as possible melanoma, including 64 histopathologically diagnosed as melanoma. These locations are carried through to the histopathological diagnosis, permitting a spatially localized comparison with the corresponding spectrum. The variations in spectra between groups of lesions with different diagnoses are examined and reduced to features suitable for discriminant analysis. A classifier distinguishing between benign and malignant lesions performs with sensitivity/ specificity of between 64/69% and 72/78%. Classifiers between pairs of the group common nevus, dysplastic nevus, in situ melanoma, and invasive melanoma show better or similar performance than the benign/malignant classifier, and analysis provides evidence that different spectral features are needed for each pair of groups. This indicates that multiple discriminant systems are likely to be required to distinguish between melanoma and similar lesions

Toward the discrimination of early melanoma from common and dysplastic nevus using fiber optic diffuse reflectance spectroscopy

We describe a study of the discrimination of early melanoma from common and dysplastic nevus using fiber optic diffuse reflectance spectroscopy. Diffuse reflectance spectra in the wavelength range 550 to 1000 nm are obtained using 400-µm core multimode fibers arranged in a six-illumination-around-one- collection geometry with a single fiber-fiber spacing of 470 µm. Spectra are collected at specific locations on 120 pigmented lesions selected by clinicians as possible melanoma, including 64 histopathologically diagnosed as melanoma. These locations are carried through to the histopathological diagnosis, permitting a spatially localized comparison with the corresponding spectrum. The variations in spectra between groups of lesions with different diagnoses are examined and reduced to features suitable for discriminant analysis. A classifier distinguishing between benign and malignant lesions performs with sensitivity/ specificity of between 64/69% and 72/78%. Classifiers between pairs of the group common nevus, dysplastic nevus, in situ melanoma, and invasive melanoma show better or similar performance than the benign/malignant classifier, and analysis provides evidence that different spectral features are needed for each pair of groups. This indicates that multiple discriminant systems are likely to be required to distinguish between melanoma and similar lesions